Using fuzzy logic to handle the semantic descriptions of music in a content-based retrieval system

This paper explores the potential use of fuzzy logic for semantic music recommendation. We show that a set of affective/emotive, structural and kinaesthetic descriptors can be used to formulate a query which allows the retrieval of intended music. A semantic music recommendation system was built, based on an elaborate study of potential users and an analysis of the semantic descriptors that best characterize the user’s understanding of music. Significant relationships between expressive and structural semantic descriptions of music were found. Fuzzy logic was then applied to handle the quality ratings associated with the semantic descriptions. A working semantic music recommendation system was tested and evaluated. Real-world testing revealed high user satisfaction

Using fuzzy logic to handle the users' semantic descriptions in a music retrieval system

This paper provides an investigation of the potential application of fuzzy logic to semantic music recommendation. We show that a set of affective/emotive, structural and kinaesthetic descriptors can be used to formulate a query which allows the retrieval of intended music. A semantic music recommendation system was built, based on an elaborate study of potential users of music information retrieval systems. In this study analysis was made of the descriptors that best characterize the user's understanding of music. Significant relationships between expressive and structural descriptions of music were found. A straightforward fuzzy logic methodology was then applied to handle the quality ratings associated with the descriptions. Rigorous real-world testing of the semantic music recommendation system revealed high user satisfaction

Music Information Retrieval. Conceptual Framework, Annotation and User Behaviour

Interactieve systemen, die automatische toegang tot muzikale informatie mogelijk maken, intrigeren zowat iedereen die in muziek geïnteresseerd is. Het probleem waar allen mee geconfronteerd worden is de moeilijke toegankelijkheid ervan. Binnen de onderzoeksgemeenschap die zich concentreert op de ontwikkeling van nieuwe methoden om informatie over muziek op te zoeken (Music Information Retrieval) wordt er veel aandacht besteed aan de op inhoud gebaseerde extractie van muziek. Dergelijk onderzoek is gericht op technieken die informatie onttrekken aan het geluidsignaal zelf. Hoewel de benadering van muziek als akoestisch fenomeen belangrijke perspectieven biedt is er tot op heden geen heldere connectie tussen de akoestische, perceptuele en cognitieve kenmerken van het geluid. In het proefschrift worden de resultaten beschreven van een muziekwetenschappelijk onderzoek uitgevoerd in het perspectief van gebruiksvriendelijke ‘Music Information Retrieval’-toepassingen. Doel van deze studie is bij te dragen tot de opheffing van het gebrek aan overeenstemming tussen de automatische descriptie van muziek en de manier waarop mensen muziek ervaren en interpreteren. Het onderzoek evolueert doorheen een theoretische, empirische en praktische context. De theoretische benadering omvat de definiëring van een conceptueel kader gebaseerd op methodologische beschouwingen betreffende manuele annotatie van muziek. Het is de eerste studie binnen het onderzoeksdomein die een conceptueel kader naar voor brengt dat de integratie ondersteunt van resultaten van verschillende onderzoeksgemeenschappen. Daarbij wordt rekening gehouden met de wederzijdse relaties tussen muziek als akoestisch én als expressief fenomeen. Er wordt een nieuwe methodologie voor manuele annotatie voorgesteld en getest. De resultaten tonen aan dat manuele annotatie een belangrijke schakel is voor de ontwikkeling en verfijning van algoritmes voor ‘Music Information Retrieval’-systemen. Het empirisch onderzoek omvat twee studies die uitzonderlijk zijn qua opzet en schaal. De eerste studie richt zich naar spontaan gedrag van de muziekliefhebber: enerzijds bij het zoeken naar muziek en anderzijds bij de vocale imitatie van muziek. Het tweede onderzoek resulteert in de definitie van het globale profiel van de potentiële gebruiker van interactieve systemen alsook van de manier waarop deze gebruiker muzikale kwaliteiten beschrijft. Ten slotte worden er drie praktische toepassingen voorgesteld die op basis van het theoretische kader en de empirische onderzoeksresultaten tot stand gekomen zijn: een voorbeeld van een gebruikersomgeving, een geannoteerde gegevensbank beschikbaar voor verder onderzoek, en een prototype van een systeem dat muziek aanbeveelt op basis van emotie, klank en beweging. In dit proefschrift wordt aangetoond dat onderzoek naar de muzikale achtergrond van gebruikers, hun interesses en hun perceptie van muziek belangrijk potentieel heeft voor de ontwikkeling van volledig geïntegreerde ‘Music Information Retrieval’-systemen. Er wordt aangetoond dat door het inschatten van de interpretatieve consequenties van descriptie van muziek door gebruikers, een analytische benadering ontwikkeld kan worden die focust op de wisselwerking tussen de muziekliefhebber en muziek als fysisch object. Het conceptueel kader dat wordt voorgesteld vormt de basis voor de ontwikkeling van een taxonomisch systeem dat toelaat semantische concepten te linken aan de muzikale inhoud

Music information retrieval: conceptuel framework, annotation and user behaviour

Understanding music is a process both based on and influenced by the knowledge and experience of the listener. Although content-based music retrieval has been given increasing attention in recent years, much of the research still focuses on bottom-up retrieval techniques. In order to make a music information retrieval system appealing and useful to the user, more effort should be spent on constructing systems that both operate directly on the encoding of the physical energy of music and are flexible with respect to users’ experiences. This thesis is based on a user-centred approach, taking into account the mutual relationship between music as an acoustic phenomenon and as an expressive phenomenon. The issues it addresses are: the lack of a conceptual framework, the shortage of annotated musical audio databases, the lack of understanding of the behaviour of system users and shortage of user-dependent knowledge with respect to high-level features of music. In the theoretical part of this thesis, a conceptual framework for content-based music information retrieval is defined. The proposed conceptual framework - the first of its kind - is conceived as a coordinating structure between the automatic description of low-level music content, and the description of high-level content by the system users. A general framework for the manual annotation of musical audio is outlined as well. A new methodology for the manual annotation of musical audio is introduced and tested in case studies. The results from these studies show that manually annotated music files can be of great help in the development of accurate analysis tools for music information retrieval. Empirical investigation is the foundation on which the aforementioned theoretical framework is built. Two elaborate studies involving different experimental issues are presented. In the first study, elements of signification related to spontaneous user behaviour are clarified. In the second study, a global profile of music information retrieval system users is given and their description of high-level content is discussed. This study has uncovered relationships between the users’ demographical background and their perception of expressive and structural features of music. Such a multi-level approach is exceptional as it included a large sample of the population of real users of interactive music systems. Tests have shown that the findings of this study are representative of the targeted population. Finally, the multi-purpose material provided by the theoretical background and the results from empirical investigations are put into practice in three music information retrieval applications: a prototype of a user interface based on a taxonomy, an annotated database of experimental findings and a prototype semantic user recommender system. Results are presented and discussed for all methods used. They show that, if reliably generated, the use of knowledge on users can significantly improve the quality of music content analysis. This thesis demonstrates that an informed knowledge of human approaches to music information retrieval provides valuable insights, which may be of particular assistance in the development of user-friendly, content-based access to digital music collections

Methodological considerations concerning manual annotation of musical audio in function of algorithm development

In research on musical audio-mining, annotated music databases are needed which allow the development of computational tools that extract from the musical audiostream the kind of high-level content that users can deal with in Music Information Retrieval (MIR) contexts. The notion of musical content, and therefore the notion of annotation, is ill-defined, however, both in the syntactic and semantic sense. As a consequence, annotation has been approached from a variety of perspectives (but mainly linguistic-symbolic oriented), and a general methodology is lacking. This paper is a step towards the definition of a general framework for manual annotation of musical audio in function of a computational approach to musical audio-mining that is based on algorithms that learn from annotated data. 1

Multimodal analysis of synchronization data from patients with dementia

Little is known about the abilities of people with dementia to synchronize bodily movements to music. The lack of non-intrusive tools that do not hinder patients, and the absence of appropriate analysis methods may explain why such investigations remain challenging. This paper discusses the development of an analysis framework for processing sensorimotor synchronization data obtained from multiple measuring devices. The data was collected during an explorative study, carried out at the University Hospital of Reims (F), involving 16 individuals with dementia. The study aimed at testing new methods and measurement tools developed to investigate sensorimotor synchronization capacities in people with dementia. An analysis framework was established for the extraction of quantity of motion and synchronization parameters from the multimodal dataset composed of sensor, audio, and video data. A user-friendly monitoring tool and analysis framework has been established and tested that holds potential to respond to the needs of complex movement data handling. The study enabled improving of the hardware and software robustness. It provides a strong framework for future experiments involving people with dementia interacting with music